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This paper will describe and review an approach to quality evaluation and improvement in early 
childhood settings in the UK which focuses primarily on enhancing the effectiveness of the 
early learning experiences of young children. The work has been undertaken within the remit 



of a major national research project called “Effective Early Learning". In its first two years of 
operation the Project has attempted to map the diversity of early learning expieriences offered to 
3 and 4 year old children within a wide range of early childhood settings in the UK. It has also 
been the mechanism for a major improvement strategy within these settings, providing the 
means by which their development may be encouraged and then tracked.The EEL approach to 
quality evaluation and improvement will be reviewed and emerging issues from the first two 
phases of the Project will be highlighted. 



Early Childhood Evaluation and Improvement in the UK 




British experience of evaluating and improving early childhood programmes is not extensive. 
There is no coordinated and coherent system of provision for young children and this has led to 
the development of a diverse and complex plethora of early childhood programmes, many of 
which are rarely monitored and evaluated at all. Added to this is the paucity of funding 
available for the development of evaluation strategies and for evaluation studies in the early 
childhood field. However, with the recent political focus on developing provision for young 
children, and the emphasis on value for money and accountability which permeates all policy 
initiatives, there has been an increase in evaluative activity. Some of the evaluative initiatives 
have a clear focus on measuring or assessing quality as part of a policy for quality inspection or 



eric'' 



2 



BEST COPY AVAILABLE 



assurance. Examples of these include the British Standaixis BS 5750 or BSEN/ISO 9000 
Scheme (BSI 1991), the OFSTED Inspection Framework (1993) and the PLA Accreditation 
Procedures (PPA 1993). Other evaluative initiatives are more geared towards quality 
improvement or total quality management, developed as part of a providers commitment 
towards raising standards in their provision. Examples of the latter include the Thomas Coram 
Self Evaluation Materials (Mooney et al 1994), the Strathclyde “Evaluating Ourselves” Project 
(Wilkinson and Stephen 1992) and the Effective Early Learning Project (Pascal, Bertram and 
Ramsden 1994a, 1994b, Pascal et al. 1995). Of all of these schemes the Effective Early 
Learning Project (EEL Project) provides the most comprehensive, extensive and well trialled 
evaluation study in early childhood ever undertaken within the UK (Williams 1995). In this 
paper 1 shall discuss what we have learned through the Project about effective evaluation and 
what are seen as the key issues faced by those concerned with developing future strategies for 
quality evaluation and improvement in early childhood services. 



The Effective Early Learning Research Project 

The Effective Early Learning (EEL) Research Project began work in May 1993 and is just 
completing its second, developmental, phase of operation. The next Phase will involve an 
extensive dissemination programme for the EEL Evaluation and Improvement process, and a 
comparative analysis of quality in different types of early childhood setting. The Project grew 
out of the urgent need for procedures to facilitate quality evaluation and improvement in the 
diverse range of settings in which under fives are being educated in the UK. It also responded 
to the lack of a substantial empirical data base on the quality and effectiveness of early learning 
offered in these settings. It focuses particularly on provision for 3 and 4 year olds as these 
children currently are in a wider range of provision than any other age group, but has 
applicability throughout the early childhood years (and even beyond). The Project is operating 
throughout the UK and is being carried out by a team of practitioner researchers, directed by 
Professor Christine Pascal and based at Worcester College of Higher Education. 

The key aims of the Project are: 



1 . To develop a cost-effective strategy to evaluate and improve the quality and effectiveness of 



early learning available to 3 and 4 year old children in a wide range of education and care 
settings across Scotland and Northern Ireland, Scotland, England and Wales. 

2. To evaluate and compare rigorously and systematically the quality of early learning provided 
in a diverse range of early childhood education and care settings across the UK. 

The Project provides a clear and targeted strategy for change and improvement which builds 
upon the existing range of provision for young children and attempts to extend the skills and 
expertise of all those who work with young children. It brings together education and care 
provision, and includes those in the voluntary, public and private sectors. It centres round the 
development and application of an innovative and cost effective set of “Evaluation and 
Improvement” procedures which may be used for training, institutional development, 
monitoring and review in all early childhood settings. The development of quantitative and 
qualitative instruments to evaluate and compare the quality of provision in different settings is 
also a key feature of the Project. 

The project, therefore, has two inter linked, and complementary elements; that of research and 
development. A main thrust of the Project’s work is to develop and improve the quality and 
effectiveness of young children’s learning. It is the experience of this development process that 
is reported in this paper. However, this process of evaluation also provides a wealth of detailed 
qualitative and quantitative data from early childhood settings across the UK. Data on such 
things as training, staff ratios, curriculum, facilities, teaching styles, interactions, daily 
programmes, planning and assessment procedures, equal opportunities, home/school 
partnership and quality control procedures have been collected which will allow a comparative 
evaluation to be made of different kinds of provision. This comparative research data is 
currently being analysed and will be published during 1996. 



Emerging Issues in Early Childhood Evaluation and Improvement 

The experience of four years development work on the Effective Early Learning Project, have 
convinced us that any quality evaluation and improvement process for early childhood should 
adhere to the following six principles of action. 
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1. Adopting a dynamic, developmental approach which views the processes of evaluation and 
improvement as inseparable. 

2. Utilising procedures which are shared, democratic and collaborative. 

3. Implementing a bottom up process which is opted imo and not imposed. 

4. Creating a rigorous, systematic and agreed evaluative framework which is implemented over 
an extended period of time. 

5. Ensuring the action is supported and has outcomes which are monitored. 

6. Aiming to develop a process which empowers and develops practitioners, parents and 
children. 

Each of these six principles throw up a key issue which has to be addressed by those who are 
looking ahead to future development. All the current approaches to quality in early childhood 
have had to tackle these six issues, some managing it more successfully than others. 



Issue 1: Adlopting a developmental approach 

Williams ( 1995) makes the distinction between approaches to quality which are dynamic and 
those which are static. Dynamic approaches are concerned with a more developmental, 
incremental approach to quality improvement. They focus their attention on aspects of practice 
which can and should be subject to on-going development and the processes by which this 
development might be brought about. Static approaches are more concerned with putting into 
place mechanisms for achieving a fixed, predetermined and defined standard of quality. They 
focus their attention on the systems and procedures which will ensure an organisation is able to 
achieve a defined standard ie. the capability to deliver, rather than exploring the actuality of 
whether these standards are achieved or not ie. the actual performance. 
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We have already stated our belief that quality is “a dynamic concept which varies with time and 
place” (Pascal and Bertram 1994b). It follows, therefore, that we feel it would be inappropriate 
to lay down fixed, static, predetermined definitions of quality. We have found that if any 
organisation is to succeed consistently, and over time, it must put into place procedures and 
processes of quality evaluation and improvement which are a permanent and normal part of its 
ongoing professional activity. To view quality as some kind of magic threshold over which a 
setting may cross, collecting a kite mark of quality on the way, and then settle back content 
would be, in our view, a grave mistake. Rather, quality should be viewed as something to be 
pursued by all providers and practitioners at every step in their career, in a positive, 
developmental and absolutely professional manner. 

We have found this developmental approach to quality to be very helpful in motivating 
practitioners and providers who have been involved in the Effective Early Learning Project. 
Working in widely different early childhood contexts, it would have been extremely difficult to 
impose on every'one a fixed, tightly defined, set of quality standards towards which they 
should all work. Some of our settings would have found the demands of this totally 
overwhelming and would have been threatened and disempowered by the whole process. 
Others would have found these quality standards to be totally irrelevant as guidance for the 
future development of the high quality provision they were already achieving, and they would 
have been demeaned by the process. Added to this was our awareness that to impose a fixed, 
static set of quality standards might have led to the impression that once these had been 
achieved, and acknowledged, then there was little more to be done. In fact, some of our most 
developed settings set themselves even greater challenges in their commitment towards 
“improving on their previous best”. 

A further mechanism to facilitate the kind of ongoing, developmental commitment we were 
trying to put in place was the intertwining of the processes of evaluation and improvement so 
that they had to be viewed as inseparable. In the Effective Early Learning Project, the 
procedures for gathering evidence and making judgmencs about what this evidence revealed, 
are inextricably linked by both personnel and techniques, to the action required for quality 
improvement. We believe that separating out these two processes leads to the dangerous 
allegation that the judgments made are inappropriate, lacking in credibility and limited in their 
understanding of the context. We can also empathise with teams of practitioners who refuse to 



comply with the suggested diagnosis of an evaluative process in which they have played no 
part. Experience has shown that separating evaluation from improvement is fatally flawed. 



Issue 2: Utilising a democratic approach 

Given our stated belief that assessing and improving quality is a subjective and value-laden 
enterprise, it therefore follows that any process for achieving this must embrace this notion at 
its heart. It must facilitate and encourage the expression of the values, preferences and opinions 
of ail those who play a part in the life of the setting. Involving the so-called “stake holders" 
(Moss and Pence 1994) in the quality evaluation and improvement process is an idea that we 
concur with wholeheartedly in terms of meaning, if not in terms of semantics. Adopting a 
“democratic’ approach requires that all those who have a close involvement in the early 
childhood setting should be invited to play a part in the development process. This means 
involving managers, practitioners, parents and, importantly, children. In this sense, the whole 
process should be something which is “done with” the participants, rather than “done to” them. 

This approach coheres with the views of Pfeffer and Coote (1991) who critiqued the existing 
“traditional”, “scientific", “managerial” and “oonsurnerist” approaches to quality which are 
generally utilised in the welfare services. They argue that each of these approaches fail because 
they do not acknowledge the important distinction between commerce and welfare. 
Alternatively, they propose a “democratic” approach which recognises the central importance of 
providing equity in services and ensuring that everyone has the opportunity for expressing their 
needs and preferences. We support this approach strongly, both philosophically and 
professionally. It acknowledges and celebrates the subjectivity of defining quality as an 
empowering and strengthening process. 

In the Effective Early Leaning Project we have tried to ensure that quality is defined by the 
shared reflections and agreement of experienced managers, practitioners, parents and children. 
The definition is validated and scrutinised for accuracy by those closest to the experience that is 
being evaluated. The Project is therefore firmly founded in democratic principles and we have 
worked hard at putting in to place a process which depiends on partnership, collaboration and 
teamwork ,( although we might discuss at some length the distribution of power within these 



relationships). Our philosophical commitment to this approach was reinforced with the hope 
that it would also enable the individual settings to become more responsive, more fir for 
purpose and empower those within them. 

Some recent quality initiatives have adopted a similar inclusive, collaborative approach - (The 
PLA Accreditation Scheme 1993 and the Strathclyde Project 1994). Others have tended to view 
the process as being more effectively carried out by an external team of “experts” who come 
into an early childhood setting and implement the quality evaluation process - (the British 
Standards Scheme 1991 and the OFSTED Inspection Scheme 1993). However, we believe that 
if ongoing quality improvement is viewed as part of a complex set of continuously evolving 
relationships between providers, children and their families, then it is crucial that approaches 
adopt a participative, collaborative mode of operation. For us, this is a key issue to be 
addressed by those concerned with developing quality. We have found that parents, children 
and practitioners need to be encouraged to work in a mutually open, honest and supportive 
partnership which is directed towards ensuring the highest quality of early learning experiences 
possible. 



Issue 3: Implementing a bottom up process 

Working with early childhood practitioners over many years has convinced us of their deep 
commitment to providing children with the best quality of early experiences possible. We have 
not found the need to coerce practitioners into improvement schemes as long as they feel it will 
help them practically to do a more effective job. In fact, we have found practitioners from all 
sectors actively seeking opportunities to become more skilful educators. The biggest complaint 
we hear is of the lack of available and appropriate professional development opportunities, and 
the difficulties they face in being able to take up what is available due to inadequate funding, 
limited time and the lack of access to expertise. We see time and again practitioners who are 
willing to give up their own time, energy and money to engage in a process of professional 
development. The pool of enthusiasm and passion for improvement displayed by the staff who 
work with young children is notable and those who are developing quality schemes can only 
benefit from this. Experience across industry through Total Quality initiatives confirm our 
experience on the Effective Early Learning Project, and reinforce our belief that quality 



improvement is most effective and powerful when it is a bottom up process into which 
participants opt because they can see its relevance, rather than a top down, management 
imposed, directive. 

However, this emphasis on quality schemes which are practitioner and provider owned and 
opted into, rather than centrally prescribed and imposed, does have a proviso. We believe 
strongly that all early childhood settings should be part of a nationally regulated and carefully 
monitored system of provision for young children, in which all providers are required to reach 
a baseline of quality before they are allowed to operate at all. Once the initial operating baseline 
is achieved, each provider should be encouraged to link into an ongoing scheme of quality 
evaluation and improvement as part of their commitment to establishing high standards across 
the sector. Thus we are arguing for top down systems to take responsibility for instituting and 
monitoring the national movement towards high quality provision for all children, and bottom 
up systems of quality improvement to provide the vehicle by which this movement is achieved 
and maintained. 



Issue 4: Developing a systematic and rigorous evaluative framework 

Any process of quality evaluation and improvement must have at its heart a clearly articulated 
framework on which to base its reflection and action. We have spoken of the growing 
consensus, which is well laid out in both DES (1990) and DoH (1991) publications. Looking 
across the spectrum of quality schemes there is a significant degree of overlap in the evaluative 
frameworks used. The OFSl ED Framework, the PLA Accreditation Scheme, the Strathclyde 
Project and the Effective Early Learning Project all look at similar “domains” or “dimensions” 
of policy and practice. These dimensions generally include management procedures, policy, 
staffing, curriculum, physical environment, interpersonal relationships, ethos and home links. 
We have found that working from this agreed evaluative framework has been enormously 
helpful in ensuring a cohesive, coherent and comprehensive response to the quality movement. 
If each early childhood provider was to work to an individual evaluative framework, further 
fragmentation and disparity of quality within the system as a whole would follow. Yet, three 
key points need to be emphasised. 



Firstly, to be effective any evaluative framework must be rigorous, systematic and based upon 
the best knowledge we have about effective teaching and learning in the early years. This will 
involve the development and utilisation of accessible and practicable techniques for gathering 
and analysing evidence on which to base the evaluation and the training of practitioners in 
employing them. At the heart of these techniques should be focussed observations of adults 
and children within a setting, but they will also include a range of other qualitative and 
quantitative methods of gathering information. The model of practitioner as researcher should 
therefore be viewed as central to the quality improvement process. 

Secondly, while the framework itself has to be robust and transferable, it also has to be flexible 
so that each element within it can be interpreted to meet the particular context in which it is 
being applied. The diverse range of early childhood settings within the UK, and the need for 
these to be responsive to the families and local community they serve, demands that there is 
room within any quality framework for it to be applied in a range of different ways. This 
flexibility should allow individual settings to offer parents real choice whilst reassuring them 
that the core elements of quality are being addressed. 

Thirdly, all those participating in the evaluative process must be aware of this quality 
framework and agree on its validity and applicability to their particular context. Where dispute 
arises as to the relevance or appropriateness of any aspect of the framework, the effectiveness 
of the whole process is threatened. The evaluative framework being used must have credibility 
and acceptance amongst all members of the organisation which is being evaluated. This 
requires good communication, time for everyone to familiarise themselves with the framework, 
and opportunities for an open dialogue about it. 

Experience has also taught us that a dip stick approach to quality evaluation and improvement 
severely limits its effectiveness. In order to obtain a comprehensive, truly representative and 
valid picture of the quality of provision in any one setting, which can be used as the basis for 
fundamental improvement which will have a lasting impact, a long term time frame has to be 
used. The Effective Early Learning Project’s evaluation and development process takes 
between 6 to 9 months to go through just one cycle of focussed development. Other schemes 
also have an extended time period for their implementation eg the Strathclyde Project took over 
1 2 months and the PLA Scheme has no time limits. We have found it is important that the 



process of qualil ; improvement is not viewed as a short, sharp blast of activity which can be 
done periodically and then put on one side. Rather, we would promote a model of on going, 
professional activity directed at a constantly rolling cycle of evaluation and improvement. In 
this way short, medium and long term goals can be planned for, and worked at systematically, 
and at a pace which individual settings can manage within the normal ebbs and flows of their 
activity. We have found this to be not only pragmatic and realistic, but also motivating for 
those involved because they feel in control. 



Issue 5: Monitoring and assessing the impact 



The issue of monitoring and assessing the impact of any quality improvement strategy is one 
which must be considered carefully, for it is full of both pitfalls and potential. With limited 
resources it is important to have some evidence that any action undertaken has achieved what it 
set out to do. Accountability and value for money are part of the climate in which we are all 
working, but they only form part of the reason for focussing on this issue. Early childhood 
educators are primarily concerned with providing children with the early learning experiences 
that will lay a sound foundation for the child s future. To get this wrong at this stage will have 
far reaching consequences for individual children and for society at large. The central purpose 
of any improvement strategy, therefore, should be the enhancement of the quality of the 
teaching and learning processes offered, and it is therefore important that we have mechanisms 
which can assess this. 

Unfortunately, assessing the quality of the processes that go on within any setting is a very 
tricky task. It is not easy to identify the constituent elements within a quality experience and to 
gather “hard” evidence about changes in these. We are only just beginning to understand the 
subtle qualitative nuances, interp>ersonal relationships and factors which constitute effective 
teaching and learning at this stage, but it is clear that these are the critical factors in determining 
a quality education or not. As a result of this lack of well developed techniques, process 
measures do not seem to carry the same attraction to those who monitor quality in early 
childhood. Outcome measures which can provide tangible and often quantifiable evidence are 
often seen, mistakenly in our view, as preferable. This is despite the fact that the outcomes of 
educational inputs in these early stages may not be evident until the child reaches maturity. 
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Y et, we would be wrong to polarise the debate. It is important that we focus our attention on 
the development of measures to assess the impact of any programme of professional 
improvement within a setting. Some of these impact measures may focus on educational 
outcomes and these would include a child’s social competence, emotional well being, 
behavioural characteristics, linguistic skills, mathematical competencies, amongst them. 
However, given the emphasis placed upon learning processes at this stage (DES 1990), we 
urgently need to develop impact measures which provide reliable and accessible evidence of the 
quality of these processes. These measures are beginning to emerge and to be utilised within 
quality improvement schemes. For example, the Effective Early Learning Project has worked 
with Professor Ferre Laevers to develop two observational assessment techniques which aim to 
measure quantitatively the quality of the learning and teaching processes. The Involvement 
Scale (Laevers 1994) focuses on the quality of the learning process and the Engagement Scale 
(Bertram 1995) focuses on the quality of the teaching process. These two structured 
observational techniques are manageable and accessible and have provided invaluable evidence 
to practitioners, providers and consumers, which may be used diagnostically and evaluatively 
to monitor developments in quality over time and to make comparisons. More work clearly 
needs to be done in this important area in developing assessment techniques, but also in 
convincing decision makers of the validity and reliability of such process measures. It is 
interestin;_ to note that amongst the quality schemes currently available to early childhood 
providers, few are really addressing the issue of monitoring the impact of improvements on the 
quality of teaching and learning at all. 



Issue 6: Aiming for empowerment 

Despite a developing national profile and the increasing acceptance of the importance of early 
childhood, those who work with young children continue to suffer from low status, a lack of 
professional acknowledgement and the perception that their work is less valuable than those 
who work with older children (DES 1990. Pascal 1993, National Commission Report 1993. 
RSA Report 1994). The effect of this is felt throughout the system and at all levels and so any 
long term strategy to improve quality in early childhood has got to tackle this issue vigorously. 
In a competitive world, resources generally follow status, visibility and power and we in early 



Ci^ildhood do badly on all these counts. It would also be disingenuous for us to pretend that 
any substantial improvement in the quality of provision for young children could be achieved 
without tackling the broader social and political context in which such provision is set. At 
present, despite a number of recent reports which have protested against the inequity in the 
funding directed at the education and care of young children, a substantial shift of resources in 
favour of early learning has not been achieved (DES 1990, National Commission Report 1993, 
RSA Report 1994, Houses of Commons 1994). 

We need to continue to campaign nationally and locally for this shift, but underpinning all of 
this should be a strategy which aims to enhance the professional competence, confidence and 
assertiveness of early childhood educators. This must feed into all initial and in service 
training, but can also be greatly facilitated through the development of approaches to quality 
improvement which celebrate and empower the practitioners themselves. We have found 
through the Effective Early Learning Project that our strategy of acknowledging the skills and 
expertise of those who work with young children, giving them the responsibility for taking 
their practice on and supporting them in this process, has had a tremendously empowering 
effect on those with whom we have worked. It has helped them become clearer and more 
articulate about their practice and their requirements for putting into place high quality early 
learning experiences for the children in their care. It has also given them the confidence and the 
evidence to start asserting their needs to the decision makers who oversee their work. It is this 
that gives us most cause for optimism in the future. At last we are beginning to realise the 
importance of our work in eariy childhood and to have high expectations of those who should 
be supporting us in carrying it out. 



In conclusion 



Putting in place high quality early learning experiences which are available for all children will 
not be achieved overnight, and nor will it come cheap. Substantial investment in quality 
provision for young children is urgently needed and long overdue. We have to be absolutely 
clear, as limited resources are made available to us, how these might most effectively be used, 
what kind of quality provision we are aiming to put in place and what the first steps are in 
achieving this. We must also make sure that we do not lose the small gains in quality we have 
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made in the current rush to expand provision quickly and at the least possible cost. The 
question of quality in early childhood is a crucial one and one which we cannot afford to get 
wrong. 



For more information contact: 

Professor Christine Pascal, 

Centre for Early Childhood Research, 
Worcester College of Higher Education, 
Henwick Grove, 

Worcester, 

WR2 6AJ, 

UK. 
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